Subjective feelings associated with expectations and rewards during risky decision-making in impulse control disorder

Impulse Control Disorder (ICD) in Parkinson’s disease is a behavioral addiction induced by dopaminergic therapies, but otherwise unclear etiology. The current study investigates the interaction of reward processing variables, dopaminergic therapy, and risky decision-making and subjective feelings in patients with versus without ICD. Patients with (n = 18) and without (n = 12) ICD performed a risky decision-making task both ‘on’ and ‘off’ standard-of-care dopaminergic therapies (the task was performed on 2 different days with the order of on and off visits randomized for each patient). During each trial of the task, participants choose between two options, a gamble or a certain reward, and reported how they felt about decision outcomes. Subjective feelings of ‘pleasure’ are differentially driven by expectations of possible outcomes in patients with, versus without ICD. While off medication, the influence of expectations about risky-decisions on subjective feelings is reduced in patients with ICD versus without ICD. While on medication, the influence of expected outcomes in patients with ICD versus without ICD becomes similar. Computational modeling of behavior supports the idea that latent decision-making factors drive subjective feelings in patients with Parkinson’s disease and that ICD status is associated with a change in the relationship between factors associated with risky behavior and subjective feelings about the experienced outcomes. Our results also suggest that dopaminergic medications modulate the impact expectations have on the participants' subjective reports. Altogether our results suggest that expectations about risky decisions may be decoupled from subjective feelings in patients with ICD, and that dopaminergic medications may reengage these circuits and increase emotional reactivity in patients with ICD.

ICD is characterized by a sudden increase in risky decisions caused by dopaminergic therapies.To test the hypothesis that patients with ICD will take more risks even when off medications, we compared risk-taking behavior in patients with ICD, versus patients without ICD, in an on-versus off-medication state.18 patients with ICD and 12 patients without ICD each completed 2 visits: one visit while on their dopaminergic medication and one visit off their dopaminergic medication with the order of on-and off-medication visits randomized for each patient.The age, gender, medication status of the first visit, and the dopaminergic medications prescribed were not significantly different between the ICD and non-ICD groups (Supplementary Table 1).
During each visit, patients completed a risky decision-making task consisting of multiple trials where they must choose between a certain reward option and a gamble option (Fig. 1).The decision to gamble occurred at the same rate in both groups (Supplemental Table 2) and was also not affected by medication state (off-medication, ICD = 47.00%,non-ICD = 46.04%,p-value = 0.9502; on-medication, ICD = 44.97%,non-ICD = 47.57%,p-value = 0.3048; ICD-on versus ICD-off, p-value = 0.7666; non-ICD-on versus non-ICD-off, p-value = 0.4023).
We fit a mixed-effects logistic regression model to participants' decision to gamble with independent variables for the expected value of the gamble option, the value of the certain reward option, and the subjective feelings rating of the previous trial, and examined the differences in model coefficients between each group.Logistic regression models were fit separately for each participant and for both the on and off visits to account for individual variations in decision-making tendencies.We then examined the resulting model coefficients from each group.The model parameters from the logistic regression models revealed that, as expected, both groups' decisions to gamble were positively influenced by the expected value of the gamble ( β 1 : ICD off = 1.9517,ICD on = 2.2426, non-ICD off = 2.3155, non-ICD on = 2.1248) and negatively influenced by the value of the certain reward ( β 2 : ICD off = −1.8869,ICD on = −2.0518,non-ICD off = −2.1320,non-ICD on = −1.9846) in both on and off medication states (Table 1).The prior trial's subjective feelings rating did not significantly influence the decision to gamble in any group.

Predictors of subjective feelings differentiate ICD-status in off-medication state
We next tested the hypothesis that the influence of objective decision-making variables on participants' subjective experience would be different in patients with a history of ICD versus those without (Table 2).During the task, participants were instructed to rate their subjective feeling of the outcome after a third of randomly selected trials.We fit Rutledge's happiness model 16 using a hierarchical Bayesian approach to behavioral data from each participants' off-medication and on-medication visits.The happiness model parameters consist of an intercept term (w 0 ) which serves as a baseline; a weight for the value of the certain reward option (w 1 ) if the certain reward Figure 1.Timeline of events during the Sure Bet Or Gamble Task.The Sure Bet or Gamble (SBORG) Task is composed of independent trials (A) that participants interact with using a game controller (B) and computer screen.(A) On each trial, participants are asked to choose between a sure bet (a single number with 100% probability if selected) and a gamble (two numbers each with a 50-50% probability if selected).Consistent feedback about choice selection and outcome is given for each trial.Randomly, with 33% probability, participants are asked about their subjective feelings on a separate ratings screen (C).Note: the orange text is shown for descriptive purposes only; participants do not see the text shown in orange.

Table 1.
Gamble choice model parameters.The "decision to gamble" on each trial was modeled as a dependent binary outcome with the independent variables being the "expected value of the gamble option", "certain reward value", and the participant's actual or imputed "subjective feeling rating on the previous trial", and a constant term "Baseline".β x -coefficients for each independent variable were fit in a logistic regression and reported.www.nature.com/scientificreports/ is chosen on a particular trial; a weight for the expected value of the gamble option (w 2 ) if the gamble is chosen in a trial; a weight for the reward prediction error (w 3 ), and a forgetting factor ( γ ) which modulates the extent to which events in past trials impact subjective feelings in the current trial.Parameters in the hierarchical model were simultaneously fit to each individual, as well as on a group level to the ICD and non-ICD groups, resulting in 2 sets of parameters: individual-level parameters which describe the behavior of each individual in the group during their off-medication visit, and group-level parameters which characterize the ICD and non-ICD groups as a whole when off their medication.Individual-level parameters were used to evaluate model fit by comparing the model's predicted ratings to participants' ratings.The model fit well to participant data (Fig. 2) with r 2 = 0.3888 for the non-ICD group and r 2 = 0.1761 for the ICD groups' off-medication visit.To examine differences between groups, we compared the group-level parameter sets by evaluating the effect size (Cohen's d), 95% highest density intervals (HDI), and credible values between each groups' posterior distribution.Subjective feeling (i.e., 'Happiness') model parameters for patients in the off-medication state were significantly different across ICD and non-ICD groups as observed in the posterior distributions for each parameter (Fig. 3, Table 2a).The baseline weight was larger in ICD compared to non-ICD, and the influence of the value of the certain reward and the expected value of the gamble were smaller in the ICD group.95% HDIs, which indicate which points of a distribution are most likely to contain the true value, were non-overlapping for the baseline weight (w 0 , ICD HDI: [−0.860, −0.407], non-ICD HDI: [−1.379, −0.931]), the certain reward term weight (w 1 , ICD HDI: [0.098, 0.178], non-ICD HDI: [0.196, 0.289]), and the weight for the expected value of chosen gambles (w 2 , ICD HDI: [0.058, 0.157], non-ICD HDI: [0.176, 0.278]).To gain a better understanding of these differences, we report the posterior distribution of differences between the ICD and non-ICD group when off medication which show a clear separation between how the two groups weigh the certain reward, the Table 2. Happiness model group-level parameter comparisons: ICD versus non-ICD.The 95% highest density interval (HDI) for each parameter's posterior distribution is reported to summarize each distribution.Cohen's d was used to measure the difference between posterior distributions in each group and provide an estimate the effect size between ICD and non-ICD groups.The 95% HDI and credible differences for the posterior distribution of differences between ICD-off versus non-ICD-off (3a) and ICD-on versus non-ICD-on (3b) are reported.2a).
In both the on-and off-medication state, the weight of the reward prediction error term on subjective feelings trended towards significant differences between the ICD versus non-ICD groups irrespective of the medication status.Notably, the difference approached significance, but did not meet strict criteria.In both the on-and off-state, patients with ICD may show less influence of the reward prediction error feedback signal compared

Dopaminergic medications do not differentially influence predictors of subjective experience in ICD and non-ICD groups
The expected value of chosen gambles, collection of certain rewards, and reward prediction errors following chosen gambles are hypothesized to engage or be affected by the dopaminergic system; thus, we hypothesized that the impact of these variables on subjective experience would be modulated by dopaminergic medications used to treat PD symptoms (Table 2b).

Discussion
ICD is a behavioral addiction disorder caused by dopaminergic action 3,22 , and dopaminergic systems are engaged by reward expectation, reward feedback, and associated subjective feelings 18,[23][24][25] .Thus, we sought to understand the impact dopaminergic medications and ICD status may have on decisions to take risks and subjective feelings associated with these actions.In our experimental setting, the rate of risky decisions was not different across ICD and non-ICD groups nor affected by the medication state (Supplemental Table 2).However, the influence of expectations and reward processing variables on subjective feelings significantly differed in the ICD group compared to the non-ICD group when off medication.When both groups were on medication, this influence no longer differed significantly.Notably, the influence of feedback (i.e., the reward prediction error) appears diminished in patients with ICD, regardless of medication state; however, the potential differences in this parameter were borderline and did not meet a strict 95% HDI statistical threshold.Altogether, our results suggest that patients with ICD may be in a predisposed state where risky choices and dopaminergic medications express a differential influence on emotional states compared to patients without ICD.Our results are consistent with the hypothesis that the impact of ICD induced by dopamine receptor agonistst may reveal enhanced emotional reactivity associated with increased risky behavior driven by expectations and less influenced by consequences.Further work is needed to determine whether there is a statistically significant difference in the influence of Table 3. Happiness model group-level parameter comparisons: On versus Off medication.Participants' 'happiness' with their decision outcomes were modeled as the dependent variables using Eq. 1. Parameter weight estimates for each group (ICD and non-ICD) and medication state (on and off) were determined using hierarchichal bayesian modeling (Table 2).Cohen's d effect sizes and highest density intervals (95% HDI) for comparisons across non-ICD-on versus non-ICD-off (3a) or ICD-on versus ICD-off (3b) with corresponding credible differences are reported.www.nature.com/scientificreports/reward prediction errors across patients with versus without ICD.And, future work should explore whether a computational psychiatric approach to estimating patients' emotional reactivity to gambling behaviors could be used as a prognostic biomarker for patients at increased risk of developing ICD or perhaps other addiction disorders.
We did not find clear model evidence to support a difference in parameters between the on and off medication states within the ICD or non-ICD group.However in the ICD group, we observed that the probability that the parameters are different at w 0 = 92.27%, w 1 = 97.01%,w 2 = 96.09%,which suggest a strong likelihood of an actual difference between the two groups.This, and the fact that the difference in parameters in the on-vs offmedication states had large Cohen's d effect sizes implies that replicating the study with a larger sample size is warranted to gain confidence in the signifcance of these results.
The observation that patients with ICD take an equal number of risks as patients without ICD (independent of medication state) is consistent with our experimental design and prior work 26 , though seemingly contradictory to expectation given the ICD phenotype 3,22,26 .On any given trial there is a 50% chance that the gamble option is rationally the better option.That participants chose to gamble slightly less than 50% of the time is consistent with the general phenomena of humans being risk averse 27,28 .We conducted further analysis on trials where the gamble option was not clearly optimal, where the value of the certain reward option exceeded the expected value of the gamble option (Supplementary Table 3).However, we did not observe any significant differences in the rate at which each group chose to gamble.Notably, participants experienced a number of conditions where either the gamble or the sure bet are clearly the best choice (e.g., sure bet option is greater than either gamble outcome or gamble outcomes are both better than the sure bet option).These were included to allow a control for participants' behavior -to determine whether they continue to choose rational outcomes throughout the task.These 'control trials' may have altered the context of the more conflicting gamble versus sure bet trials or may have given participants more experience with the gamble options and thus a better experience-based understanding of the true nature of the 50/50 gamble.Either of these explanations may explain why participants do not express a gambling behavior effect in our study.However, our design and the resulting lack of gambling behavior effects enable us to investigate and report an effect on subjective feelings that may otherwise had been confounded with variable rates of gambling behavior.Our computational psychiatric approach 20,21 and utilization of Rutledge and colleagues' computational model of happiness [16][17][18] allowed for a more precise investigation into factors that influence dynamic changes in subjective feelings associated with risky choices in the context of a behavioral addiction disorder.Dopaminergic medications are expected to modulate how reward-related variables are processed by dopaminergic systems including the role those systems may play in the generation of emotion.Indeed, when patients with ICD are on their prescribed dopaminergic medications, the influence of the expected value of the gamble on their feelings increases, as does the influence of the value of the certain reward.This result is consistent with dopaminergic medications increasing emotional reactivity in the ICD group, as the increase in the weight values indicate patients become more sensitive to the expected values.In patients without ICD, these parameters saw little change between medication states, indicating dopaminergic medication did not influence how much certain reward and expected value of the gamble influenced their feelings.Patients without ICD may be in a state that diminishes or prevents the effects of the medication observed in the ICD group.The evaluation of model fit revealed that the model exhibited comparatively poorer fit in the ICD off medication group (r 2 = 0.1761) when compared to the other groups (ICD On = 0.2663, Non-ICD Off = 0.3890, Non-ICD On = 0.3699).This suggests that when off medication, the ICD cohort may experience subjective feelings that are less predictable by these factors.Alternatively, this cohort may have decreased emotional awareness and were less accurately reporting their true feelings during the task when off medication.However, additional investigation may be of interest to explore this specific possibility in greater depth as the task and model were not specifically designed to disentangle this possibility.The increased volatility observed in subjective ratings when off medication as well as the increased valuation of chosen options when on medication could potentially signify underlying neurobiological differences that may contribute to the development of ICD and may be used to detect PD patients at risk of developing ICD prior to the initiation of medication regimens.
In the present experimental context computational modeling revealed subtle differences that would otherwise be difficult to detect.This suggests potential for future work to develop a behavioral screening approach 15 .The observed differences in parameters between the ICD and non-ICD groups in the off-medication state, along with their convergence when on medication, offer promising indications that a task of this nature holds potential for distinguishing between individuals susceptible to developing ICD and those less likely to do so.The participants in this study have already experienced a standard of care approach to determine their medication strategy.In this process these patients have in the past or were currently positive for ICD.Thus, we cannot determine whether our results for patient in the off-medication state are reflective of their predisposition or if these results reflect the consequences of having already experienced an ICD-inducing medication.Further study is needed to examine whether the use of model parameters can be used to predict if a patient is prone to developing ICD following the prescription of specific dopaminergic medications.The task we use is short, only 30 min, and can likely be significantly shortened.Development of such a screening tool would be a valuable addition to other more complex multi-session interactions or neuroimaging based approaches 29,30 .More work is needed before such a tool could be implemented, but our results suggest potential utility in using objective measures of moment-tomoment changes in subjective feelings as expressed through task behavior.
There is a yet unclear and complex relationship between reward processing variables, associated subjective feelings, the dopaminergic system, and disorders like PD and ICD.Through computational modeling of risky decisions, we identify quantitative changes in the influences of reward processing on subjective feelings and how these systems may be altered by dopaminergic medications and ICD.Our results are consistent with risk taking in patients with ICD being a form of significantly altered behavior with associated changes in subjective www.nature.com/scientificreports/experience that is affected by stimuli and interventions that engage the dopaminergic system.Our results also suggest that a computational psychiatric approach may be able to identify patients at risk for developing ICD or perhaps other addiction disorders, but more work is needed before this conclusion can be reached.The present study was conducted at a single center; it is possible this introduces bias into our study that we cannot measure or detect.Similarly, a single-center study places a limit on sample size; though, our sample is comparable to previous studies 26 .Prior to data collection, a power analysis was conducted on preliminary data from a prior study involving the same patient population (i.e., patient with PD with and without ICD) performing a risky decisionmaking task to determine an estimate of an appropriate sample size.A sample size of 30 was determined to be sufficient to detect large effects.However, the present data are novel in the combination of patient population, experimental manipulations, and specific behavioral tasks and analyses.A larger sample size in future work may yield more insight into the processes involved in relating behavior to subjective feelings and individual differences to be observed in individuals with an addiction disorder.Nonetheless, our results highlight the impact computational precision can have in aiding our understanding of complex, dynamic behaviors associated with risk taking, reward processing, and subjective experience in psychiatric conditions like ICD and provide evidence more generally that latent variables that characterize subtle dynamic changes in experience may be used to better understand the mechanisms underlying human experience and behavior.

Figure 2 .
Figure 2. Happiness model performance: The happiness model was fit using hierarchical Bayesian analysis to estimate both individual-level and group-level parameters simultaneously.The model's predicted ratings appeared to follow participants' actual ratings.(A-H) Selected non-ICD and ICD individual participants' ratings and the model's predicted ratings are plotted for their on-medication visit (A, C) and off-medication visit (E, G).The resulting correlation between the actual and predicted ratings are shown in (B, D, F, H).Mean ratings and model predicted ratings across subjects in each group: (I) Non-ICD Off: r 2 = 0.3888, (J) ICD Off: r 2 = 0.1759, (K) Non-ICD On: r 2 = 0.3698, (L) ICD On: r 2 = 0.2663.

Figure 3 .
Figure 3.Posterior distributions over group-level happiness model parameters.Each subplot displays the distributions for a distinct parameter: the intercept (w 0 ), the value of the certain reward when chosen (w 1 ), the expected value of the gamble option when chosen (w 2 ), the reward prediction error (w 3 ), and the forgetting factor (γ).The colored lines below each distribution mark the 95% highest density intervals for each group, and are colored accordingly.Individual parameter means estimated by the model are marked by points below the posterior distributions.

Figure 4 .
Figure 4. Difference of means.To compare how the ICD and non-ICD groups differ, we report the difference between the ICD and non-ICD groups' posterior distributions when Off medication (Row A) and when On medication (Row B).For the effect of dopaminergic medication on each group, we report the difference of means between on and off medication posterior distributions for the non-ICD group (Row C) and ICD group (Row D). 95% highest density intervals are marked by a horizontal line at the base of each distribution.